PATENT 



Insertion of Sound Segments into a Voice Channe l 
of a Communicatio n Device 

Inventors: Jeffrey S. Malkin and Jeffrey S. Kirschner 

Related Applications 

[0001] This application claims a priority benefit under 35 USC § 1 19 of U.S. Paten 
Application 60/425,807, filed November 12, 2002, the contents of winch are herein incorporated 

by reference. 

Background of th e Invention 

1 Field of Invention 

[0002] The present invention relates to communication devices that establish a voice 
channel for the transmission of voice communications. 

2. Description of the R elated Arts 

[0003] Mobile communication devices, e.g., wireless or cordless phones, that establish a 
voice channel for transmission of voice communications include, for example, conventional 
telephones and cellular phones. Once these mobile communication devices establish a voice 
channel, sound segments from within the devices themselves or other independent sources 
cannot be mixed into that established voice channel (other than those sounds transmitted 
conventionally through the built-in microphone and dial tones generated by the device keypad) 
Specifically, unique sound segments (or sound bytes) that are stored in data chips or on an 
analog medium on the device itself cannot be mixed into the voice channel because separate 
chips or chip sets process the voice channel from the data or analog applications. This limits 
functionality, flexibility, and enjoyment of use of the mobile communication device during a 

conversation over these devices. 
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[ 0004 ] Therefore, there is a need for a communication system and method that allows for 
mixing a sound segment (or sound byte) represented by digital data or an analog signal directly 
into an established voice channel between two mobile devices where the sound segment is 
independent of the voice information. 

Summary of the Invention 



[ 0005 ] The present invention includes an electronic device and/or chip set for use with a 
wireless communication device that establishes a voice channel (audio transmission) for 
communications. The device may be external to the communication device or may be integrated 

within the communication device. 



[ 0006 ] In one embodiment, the present invention includes a conductor unit that is 
configured for providing recorded sound segments (audio that may be digital data or an analog 
signal) within a voice channel by selecting recorded audio to be played, injecting the recorded 
audio into the voice channel, and outputting the combination of the voice from the 
communication devices’ built-in microphone and the recorded- audio so that it can be heard by 



both the sender and receiver. In another embodiment, the conductor unit may also be configured 
for recording and storing audio sound segments as discrete sound samples that can be selected, 
triggering selection of the discrete sound samples, and mixing and outputting the discrete sound 
samples into a voice or audio transmission of the voice communication device. In yet another 
embodiment, the conductor unit is configured for inserting sound segments (e.g., recorded audio 
data) into an audio transmission established by a voice communication device, the system 
including a selection mechanism that connects with a storage element to select a sound segment 
from storage, and a mixer that receives the sound segment and mixes it with available audio in a 

voice transmission. 
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[ 0007 ] The features and advantages described in the specification are not all inclusive and, 
in particular, many additional features and advantages will be apparent to one of ordinary skill in 
the art in view of the drawings, specification, and claims. Moreover, it should be noted that the 
language used in the specification has been principally selected for readability and instructional 
purposes, and may not have been selected to delineate or circumscribe the inventive subject 

matter. 



Brief Description of the D rawings 

[ 0008 ] The invention has other advantages and features which will be more readily 
apparent from the following detailed description of the invention and the appended claims, when 
taken in conjunction with the accompanying drawings, in which: 

[ 0009 ] Figure la illustrates a conductor unit that is external to a communication device m 
accordance with one embodiment of the present invention. 

[ 0010 ] Figure lb illustrates an exploded view of a conductor unit that is internal to a 
communication device in accordance with one embodiment of the present invention. 

[ 0011 ] Figure 2 illustrates a system configuration of a conductor unit in accordance with 

one embodiment of the present invention. 

[ 0012 ] Figure 3 illustrates a flow diagram of a process for mtroducing and outputting a 
sound segment within a voice channel in accordance with one embodiment of the present 



invention. 

[ 0013 ] Figure 4a illustrates a How diagram of a process of retrieving a sound segment from 
a network for storage in a conductor unit in accordance with one embodiment of the present 

invention. 
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[ 0014 ] Figure 4b illustrates a flow diagram of a process for recording a sound segment 
from an external source for storage in a conductor umt in accordance with one embodiment of 

the present invention. 



retailed Description of the Invention 

[ 0015 ] The present invention includes a software and/or hardware system that includes 
recording and storing and .0 data (digital or analog) for introduction into a voice channel (e.g„ a 
phone call) established between two or more communication devices (e.g„ voice transmission 
communication devices). In one embodiment, the communication device is a mobile 
communication device, e.g., a cellular phone, a cordless phone, or a wireless communicator. The 
audio data may be referred to as sound segments and can be pre-stored in a storage system for 
later retrieval. Alternatively, the audto data may be recorded and stored from an external source, 
for example, downloaded through a network (e.g., the Internet) or media device (e.g., an MP3 
player, a tape device, a CD player, or DVD player). In yet another embodiment, the audio data 
(or sound segment) is simply received from the media device and passed through the system and 
mixed with the voice channel. Further, it is noted that the audio data may be in any audio file 
format, including analog, audio interchange file format, audio video interleave, WAVE, 
broadcast WAVE, MPEG-2, AU, AES-31, MP3, MP4, PCM, Dolby Digital, DTS, Real Audio, 



Sony Dynamic Digital Sound, and the like. 

[ 0016 ] Looking first at Figure la, it illustrates a voice channelsound insertion system 100 
in accordance with one embodtmen. of the present invent, on. The voice channel sound insertion 
system 100 includes a conductor unit 1 10, a communication device (e.g., a phone) 120, and an 
audio device 130, which is optional. The conductor umt 1 10 includes an input port 1 15 and a 
communications device interface ( CDI ) 125. 



Case 23426-07345 



4 



23426/07345/DOCS/ 1 3691 78.3 




PATENT 



[ 0017 ] The conductor unit 1 10 provides a source of sound segments (e.g., analog or digital 
audio data) that may be introduced into a voice channel that carries voice data (or voice stream). 

I, is noted that in Figure la. the conductor unit 1 10 is shown as a stand-alone device that couples 
with a communication device 120 through the CDI 125. The conductor unit 110 may also 
include the input port 1 1 5 (e.g., an audio (o, stereo) jack, a Universal Serial Bus (“USB) port, or 
a communication transceiver such as a wireless LAN or Bluetooth transceiver) that is configured 
to receive audio from the audio device 130 (e.g., an MP3 player, a tape device, a network 
computer, a personal computer, a handheld computer, a CD player, or a DVD player). Audio 
received from the audio device 130 can be either passed through the conductor unit 1 10 for 
immediate mixing into the voice channel or stored as a sound segment in the conductor unit 1 10 
for later recall and mixing into the voice channel. 

[ 0018 ] The communication device 120 is a conventional device that establishes (or is used 
for) a voice channel, e.g., a telephone, cellular phone, wireless communicator, or Internet 
Protocol phone. The CDI 125 is a communication interface for coupling ihe conductor unit 1 10 
with the communication device 120. 

[ 0019 ] Figure lb illustrates a second embodiment of a voice channel sound insertion 
system 101 having an exploded view of the conductor unit 1 10 that is internal to the 
communication device 120 in accordance with one embodiment of the present invention. In this 
embodiment, the conductor unit 1 10 may be configured to reside within a communication device 

120 that establishes a voice channel. For example, the conductor unit 1 10 may be a self- 

contained chip or chip set that is configured to connect with the other communication chips or 

chip sets in the communication device. 
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[ 0020 ] Alternatively, the functionality of the conductor unit 1 10 may be integrated into a 
preexisting chip or chip set of the communication device 120. Moreover, if a voice and data chip 
are closely integrated or merge with regard to functionality and design, the principles of the 
present invention allow for another embodiment in which the functionality of the conductor unit 
may be comprised of a software application or firmware that functions in conjunction with that 
chip, and if necessary other components such as a microcontroller or microprocessor, to insert 

audio data into a voice channe 1. 

[ 0021 ] Figure 2 illustrates a system configuration 200 of the conductor unit 1 1 0 in 
accordance with one embodiment of the present invention. The conductor unit 1 10 includes a 
microcontroller 205, a storage 210 (e.g., an EEPROM, a flash memory, an optical or magnetic 
disk drive, or a battery backed SRAM or DRAM), a sound input/encoder 215, a sound decoder 
220, a mixer 225, a display 230, a power supply 235 (e.g., battery), one or more sound trigger 
mechanisms 240, function controls 245, and an ON/OFF switch 250. The function controls 245 
include particular functions that are specific to the device, such as recording new sound 
segments. 

[ 0022 ] Generally, the microcontroller 205 couples with the storage 210, the display 230, 
the power supply 235, the sound input/encoder 215, the sound decoder 220, the sound trigger 
mechanisms 240, the function controls 245, and the ON/OFF switch 250. The sound decoder 
220 couples with the sound input/encoder 215, the sound decoder 220, and the mixer 225. The 
mixer 225 also couples with a volume control 255, the communication device 120 (e.g., a 
cellular phone is illustrated in Figure 2) and an optional microphone extension 260. 

[ 0023 ] The microcontroller 205 is a conventional microcontroller that executes instructions 
and provides functiorality for the conductor. For example, the microcontroller 205 executes 
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instructions for the processes illustrated in Figures 3, 4a, and 4b. For example, Figure 3 
illustrates a flow diagram of a process for introducing and outputting a sound segment within a 
voice channel in accordance with one embodiment of the present invention. In addition, Figure 
4a illustrates a flow diagram of a process of retrieving a sound segment from a network for 
storage in a conductor unit in accordance with one embodiment of the present invention. Figure 
4b illustrates a flow diagram of a process for recording a sound segment from an external source 
for storage in a conductor unit in accordance with one embodiment of the present invention. The 
processes in these Figures will be further described below. 

[ 0024 ] The microcontroller 205 also provides functionality for other operations. For 
example, the microcontroller 205 functions with the sound input/encoder 215 to operate on an 
analog or digital data input to receive the data (e.g., audio, text, graphic, etc.) and prepare it for 
storage in the storage 210 or contemporaneously mix it with a voice channel through the mixer 
225 (e.g., via the sound decoder 220). As another example, the microcontroller 205 functions 
with the sound decoder 220 to retrieve stored data from the storage 210 and process the stored 
data for mixing with the voice channel through the mixer 225. 

[ 0025 ] The encoder 2 1 5 serves to convert the incoming audio information from its native 
format into the proper format to be stored within the conductor unit 1 10, or to be mixed with the 
voice program. For example, the encoder 215 can convert analog audio into the PCM format, or 
WAVE format to PCM, etc. The decoder 220 functions to provide a converse of the encoder 215 
- it converts the stored information from the format used by the storage 210 into the proper 
format to be mixed into the voice channel. For example, the decoder 220 would convert PCM 
data to MP3, PCM to analog, MP3 to analog, etc. 
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[00261 The microcontroller 205 may also be configured to function with the storage 210 to 
provide data and database storage functions. For example, storing the input data may also 
include setting up and creating categories in some other catalog or database format that provtdes 
organization for the data. This allows for easy retrieval and organization, for example, through 
menus and files visible on the display 230 and controlled through the function controls 245 
and/or the trigger mechanisms 240, all of which function with the microcontroller 205. It is 
noted that the data in the storage 2 10 may be stored in analog or digital format and may be 
compressed to allow for greater storage capacity. Further, the storage 210 may also be used for 
storing data such as pictures, graphics, and the like. 

[00271 The microcontroller 205 is also configured to function in conjunction with the 
trigger mechanisms 240 and the function controls 245. The trigger mechanisms 240 can be 
• switches that allow for selection of sound segments from the storage. The selection options may 
be viewable to a user through the display. The trigger mechanisms 240 can be dedicated buttons, 
switches, or dials. Alternatively, the trigger mechanisms 240 can also be configured to work 
with established triggers, for example, a keypad or number pad on a communication device 120. 
[ 0028 ] In yet another embodiment, the trigger mechanism 240 can be a touch screen that 
may integrated with the display 230 or on a display of the communication device 120. In 
addition, the trigger mechanism 240 may also be a graffiti or handwriting recognition area 
integrated in with the display area 230 or with the communication device 120, e.g„ a graffiti 
section on a on a personal information device (e.g., a PDA) having voice channel capability. The 
microcontroller 205 may also. provide functionality to control power levels in the conductor, e.g., 
shutting it on or off if the device is not in use for some predetermined time. The function 
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controls 245 may be configured to provide user functionality such as “Repeat Selection”, “Set 
Selection” (e.g., “sets” of files to play), auto power on/off, and the like. 

[00291 The mixer 225 allows for injection of a selected sound segment into a voice 
channel. Specifically, the mixer 225 adds two or more audio channels together and merges them 
into a signal stream. For example, once a voice channel is established, a user may inject a sound 
segment to be incorporated into that voice channel. The mixed sound segment and voice data 
gets output through a speaker in a communication device receiving the sound from the sending 
communication device 120. Note that in the transmitting phone, the mixed signal is transmitted 

into the voice channel. In one embodiment, the mixer functions as an analog device. In this 

embodiment, audio out of the storage 210 via the decoder 220 would be analog, and would add 
with the user's voice, which is also analog, in the mixer 225. In this embodiment, mixing may be 
done through, for example, “forced summing” or “summing amplifier.” 

[ 0030 ] In the forced summing process, two or more analog signals are simply isolated from 
each other with resistors and tied together. This method creates a single stream output, but may 
add some noise to the circuit. In addition, in this embodiment signal sources often are of similar 
electrical impedance so as not to interfere with each other. In the summing amplifier process, 
two or more analog signals are resistively summed at the high- impedance inverting input of an 
audio amplifier. The high- impedance input provides near total isolation between the signal 

sources. This method allows the noiseless summing of various types of signals. 

[ 0031 ] In an alternative embodiment, if the audio data is signals from a digital domain, 
mixing of the data is through conventional digital data mixing techniques using an appropriately 
configured microprocessor or digital signal processor (“DSP"). In an example of this 
embodiment, a device mixing digital data would take two or more audio data streams and 
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perform the necessary mathematical computations to create a new data stream containing both 
sets of information. 

[ 0032 ] Figure 3 illustrates a flow diagram of a process for introducing and outputting a 
sound segment within a voice channel in accordance with one embodiment of the present 
invention. For purposes of an illustrative example, the process will be described with regard to a 
phone-type communication device. 

[ 0033 ] Once the process starts 305, a user (e.g., a first party) may initially establish 310 a 
phone call using the communication device 120 between that user and a person receiving a call 
(e.g., a second party). In this example, this may be referred to as establishing a voice channel. 
During a course of a conversation between the user and the person receiving the call, a user may 
decide to inject particular sound segments into the voice channel. To do this, the user may use 
the display screen 230 to display 315 and navigate through menus and/or lists of available sound 
segments in the storage 210. Alternatively, the sounds may be preprogrammed to be associated 
with specific trigger mechanisms 240 or a combination of two or more trigger mechanisms 240 
so that that use of the display 230 may not be necessary. 

[ 0034 ] Once the user determines which sound segment they want, this selection is received 
320 by the microprocessor 205 and it is retrieved from the storage 210. The sound segment is 
forwarded to the sound decoder 220, where it is decoded 330 and sent to the mixer 225. The 
sound segment is then injected 335, e.g., mixed, into the same voice channel as the voice 
communication. The mixed sound segment and voice communication is then output 340 through 



a microphone in the communication device 120 for the receiving user to hear through a speaker 
on that user’s communication device. The process may then end 345. Hence, if the selected 
sound segment was, for example, the first three chords of the song “Oh! Suzanna,” these chords 
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would be injected into the voice channel and heard with the voice communication by both the 
person receiving the call and the person sending the transmission. 

[ 0035 ] Figures 4a and 4b illustrate examples of additional processes that allow for 
populating the storage 210 of the conductor unit 1 10 with additional sound segments. Figure 4a 
illustrates a flow diagram of a process of retrieving a sound segment from a network for storage 
in a conductor unit in accordance with one embodiment of the present invention. The network 
may be an Internet site to which the user would access and gain access to additional pre-recorded 
sound segments for downloading into the storage 210. These sound segments may be in any 
audio data format, for example, such as those listed above. 

[ 0036 ] Generally, the process starts 405 with the user accessing 410 a web site (e.g., 

HTTP, HTTPS, FTP or other connection). The user selects 41 5 a sound segment for download 
to the conductor unit 1 10. The user may elect to associate 420 a name with the sound segment or 
associate one or more of the trigger mechanisms 240 with the sound segment. The sound 
segment is then downloaded (or transferred) 425 into storage before the process ends 430. 

[ 0037 ] It is noted that in one embodiment the user may connect to the web site through, for 
example, a web enabled communication device 120 that couples with the conductor unit 110. In 
this embodiment sound segments may be made immediately available to the conductor unit 1 10. 
Alternatively, the user may connect to the web site through a personal computer or the like, 
download the sound segment, and forward it to the conductor umt 1 10 at a later time, e.g., 
download through a USB port connection between personal computer and the CDI on the 

conductor unit 1 10. 

[ 0038 ] Figure 4b illustrates a flow diagram of a process for recording a sound segment 
from an external source for storage in a conductor unit in accordance with one embodiment of 
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the present invention. The recorded sound segment can be through a recording mechanism, e.g., 
a microphone, that is built into the conductor and that is coupled with the microprocessor, 
function controls, and storage so that the sound can be recorded and stored directly by the 
conductor. 

[ 0039 ] Generally, the process starts 435 with a user recording 440 a sound segment. 
Optionally, this recorded sound segment may be converted 445 into a digital format. In addition, 
the user may elect to associate 450 a name with the recorded sound segment or associate one or 
more of the trigger mechanisms 240 with the recorded sound segment. The sound segment is 
then downloaded (or transferred) 455 into storage before the process ends 460. 

[ 0040 ] Alternatively, external media sources such as MP3 players, CD players, tape 
players, or the like, may be a source of sound segments and may connect with the conductor unit 
1 10 through the input port 1 15, e.g., an audio (or stereo) jack. In yet another embodiment, the 
sound segments may be from a computer device, e.g., a stored audio data on a computer that 
connects to the conductor through a USB port or a communication transceiver. The sound from 
these external media sources can then be captured and stored in the storage. It is also noted that 
the sound segments from the external sources may also be passed through the conductor and 
mixed with the voice channel before being output through the speaker. In such embodiments, 
the sound segments would not need to be stored in the conductor. 

[ 0041 ] In general, the present invention beneficially provides a system and a method for 
selecting and injecting audio data into a voice channel of a communication device so that both 
the person on a receiving end of voice data communication, and the person on the transmitting 
end, hear not only the voice communication through the voice channel, but also the selected 
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audio data. Thus, a system and method in accordance with the present invention increases 
flexibility and use of the communication device and also enhances enjoyment of such a device. 
[ 0042 ] Upon reading this disclosure, those of skill in the art will appreciate still additional 
alternative structural and functional designs for a technique for integrating sound segment 
directly within a voice channel in accordance with the disclosed principles of the present 
invention. In addition, the principles described herein may also apply to inserting motion or still 
frame video having associated sound segments into a voice channel. Thus, while particular 
embodiments and applications of the present invention have been illustrated and described, it is 
to be understood that the invention is not limited to the precise construction and components 
disclosed herein and that various modifications, changes and variations which will be apparent to 
those skilled in the art may be made in the arrangement, operation and details of the method and 
apparatus of the present invention disclosed herein without departing from the spirit and scope of 
the invention as defined in the appended claims. 
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